Moving body positioning device

ABSTRACT

Provided is a moving body positioning device serving as an essential element for monitoring and tracing a moving body, which moving body positioning device uses an external monitoring camera. 
     The moving body positioning device includes a self-contained sensor group that outputs a motion velocity vector of a moving body with use of an acceleration sensor provided in the moving body, a motion velocity vector estimation processing device that measures a motion velocity vector in accordance with the output of the self-contained sensor group and outputs an output sequence of the motion velocity vector together with a measured time, a monitoring camera for externally image capturing the moving body, an image analysis processing device that analyzes the image of the monitoring camera to measure a position of feet of the moving body in the image, measures a motion velocity vector at the position of the feet, and outputs an output sequence of the motion velocity vector together with a measured time, and a motion velocity vector collation processing device that collates the output sequence of the motion velocity vector estimation processing device with the output sequence of the image analysis processing device and outputs its collation result as a TRUE or a FALSE signal.

TECHNICAL FIELD

The present invention relates to a moving body positioning deviceserving as an essential element for monitoring and tracking a movingbody, which moving body positioning device uses an external monitoringcamera.

BACKGROUND ART

A technology for measuring a position and direction specialized inwalking movements of a human being as a pedestrian, with use ofself-contained sensors (e.g. acceleration, gyro, magnetic, pressuresensors) worn on the hips, toes and the like of the human being iscalled Pedestrian Dead Reckoning (PDR), and has been well studied fromthe past (Non Patent Literature 1).

In PDR, outputs of the self-contained sensors are analyzed in view ofconstraints in the walking movements of the human being, and a motionvector or a motion velocity vector of the pedestrian is estimated one byone and are accumulated.

CITATION LIST Non Patent Literature

Non Patent Literature 1

Masakatsu Kourogi, Takashi Okuma, Takeshi Kurata, “Hokosha Nabi no tameno Jizo Sensa Moju-ru wo Mochiita Okunaisokui Shisutemu to sono Hyoka”(Indoor positioning system using self-contained sensor modules forpedestrian navigation, and its evaluation), Symposium “Mobile 08”Reviews, pp. 151-156, 2008

Non Patent Literature 2

E. Foxlin, “Pedestrian Tracking with Shoe-Mounted Inertial Sensors”,IEEE Computer Graphics and Applications, vol. 25, no. 6, pp. 38-46,2005.

Non Patent Literature 3

Tomoya Ishikawa, Thangamani Kalaivani, Masakatsu Kourogi, Andrew P. Gee,Walterio Mayol, Keechul Jung, Takeshi Kurata, “Kamera to Jizo SensaMoju-ru wo heiyou shita Intarakuthibu 3 jigen Okunai Kankyou Modera”(Interactive three-dimensional indoor environment modeler with use of acamera and self-contained sensor module), Nihon VR Gakkai KenkyuHoukoku, Vol. 14, CS-3, pp. 65-70, 2009

Non Patent Literature 4

Taniguchi, Nishio, Toriyama, Babaguchi, Hagita, “Kanshi Kamera Eizou niokeru GPS Tanmatsu Keitai Yu-za no Doutei to Tsuiseki (Identificationand tracking of GPS terminal portable user in a monitoring cameraimage), Johoshorigakkai CVIM Kenkyu Houkoku, 2006-CVIM-153, pp. 315-320,2006

SUMMARY OF INVENTION Technical Problem

In relation to technical development of PDR, the following issues are tobe further solved in order to make PDR more practical.

A first problem is that in a position/direction measurement device basedon PDR, its measurement error gradually accumulates in the course ofmotion.

Secondly, as also described in Non Patent Literature 1, with PDR, motionvelocity estimated based on individual difference between pedestriansvary by a certain degree, and it is necessary to find an individualdifference parameter to appropriately correct this variation. It ispossible to estimate an individual difference parameter by having thepedestrian carry out a calibration operation in advance; this procedureis complex however, and serves as a problem in practical use.

Thirdly, in solving the first and second problems based on the imagecaptured by the monitoring camera, a human figure inside the imagecaptured by the monitoring camera needs to be associated with the humanfigure that the PDR is tracking. However, with the techniques currentlyavailable, it is difficult to steadily achieve such an associated state.

An object of the present invention is to provide a moving bodypositioning device serving as an essential element when monitoring andtracking a moving body, which moving body positioning device isaccomplished to solve such problems and which uses an externalmonitoring camera.

Solution to Problem

In order to attain such an object, the moving body positioning deviceaccording to the present invention solves the first problem of theaccumulation of measurement error by PDR, by correcting a PDRmeasurement result by combining (i) an analysis result of an imageobtained by a monitoring camera externally provided, with (ii) the PDRmeasurement result. Namely, a human figure inside the image of themonitoring camera is made associated with the human figure that PDR isto track, and a position of PDR is corrected to a position of the humanfigure in the image of the monitoring camera.

Moreover, regarding the second problem of estimating the individualdifference parameters of a pedestrian, the moving body positioningdevice according to the present invention is capable of tracking aposition of a human figure in the image by analyzing the image of themonitoring camera, and estimating its motion velocity vector. Hence, theindividual difference parameter is estimated by analyzing this resultand PDR sensor data.

Moreover, the moving body positioning device according to the presentinvention solves the third problem of associating the human figure inthe image of the monitoring camera with the human figure that the PDRtracks by following procedures described below. Namely, first, an imageof a monitoring camera whose location relationship with a floor surfaceon which a pedestrian moves is well known is analyzed, to allow focusingon a difference image with the background. Thereafter, a human figureimage in the foreground is cut out, and positions of the head and feetof that human figure are estimated. Next, by tracking the position ofthe feet one by one, a motion velocity vector or a motion vector isestimated. This motion velocity vector or motion vector is collated witha motion velocity vector or a motion vector that the PDR outputs, toassociate the PDR output with the human figure that the monitoringcamera tracks.

As a configuration of the present invention, a moving body positioningdevice of the present invention includes: an internal observation deviceprovided to a moving body, to measure and output a motion of the movingbody; an internal observation data processing device that estimates amotion of the moving body based on the output from the internalobservation device; an external observation device that observes motionof a plurality of moving bodies; an external observation data processingdevice that estimates a motion of the moving body from an observationresult by the external observation device; and a collation processingdevice that collates an output from the internal observation dataprocessing device with that of the external observation data processingdevice, and outputs its collation result as a TRUE signal or a FALSEsignal.

More specifically, the moving body positioning device according to thepresent invention includes: a self-contained sensor group that outputs amotion velocity vector of the moving body with use of an accelerationsensor provided in the moving body; a motion velocity vector estimationprocessing device that measures a motion velocity vector based on theoutput of the self-contained sensor group, and outputs an outputsequence of the motion velocity vector together with a measured time; amonitoring camera for externally image capturing the moving bodies; animage analysis processing device that analyzes an image of themonitoring camera to measure a position of feet of the moving body inthe image to measure a motion velocity vector of that position of thefeet, and outputs an output sequence of the motion velocity vectortogether with a measured time, and a motion velocity vector collationprocessing device that collates the output sequence of the motionvelocity vector estimation processing device with the output sequence ofthe image analysis processing device, and outputs its collation resultas a TRUE signal or a FALSE signal.

Moreover, the moving body positioning device according to the presentinvention includes, as its configuration: a self-contained sensor groupthat outputs a motion vector of the moving body, the motion vector beingobtained by integrating a motion velocity vector with use of anacceleration sensor provided in the moving body; a motion vectorestimation processing device that measures a motion vector based on anoutput of the self-contained sensor group and outputs an output sequenceof the motion vector together with a measured time; a monitoring camerafor externally image capturing the moving bodies; an image analysisprocessing device that analyzes an image of the monitoring camera tomeasure a position of feet of the moving body in the image to measure amotion vector of that position of the feet, and outputs an outputsequence of the motion vector together with a measured time; and amotion vector collation processing device that collates the outputsequence of the motion vector estimation processing device with theoutput sequence of the image analysis processing device, and outputs itscollation result as a TRUE signal or a FALSE signal.

Moreover, the moving body positioning device according to the presentinvention includes, as its configuration: a self-contained sensor groupthat outputs a motion vector of the moving body, the motion vector beingobtained by integrating a motion velocity vector with use of anacceleration sensor provided in the moving body; a self-contained sensorbase motion identification processing device that identifies a movementkind of the moving body in accordance with an output of theself-contained sensor group, and outputs an identification resulttogether with a measured time; a monitoring camera for externally imagecapturing the moving bodies; an image analysis processing device thatanalyzes an image of the monitoring camera to identify the movement kindof the moving body in the image and outputs its identification resulttogether with a measured time; and the collation processing device is amovement kind collation processing device that collates the output ofthe self-contained sensor base motion identification processing devicewith the output of the image analysis processing device, and outputs itscollation result as a TRUE signal or a FALSE signal.

Moreover, in the moving body positioning device according to the presentinvention, when the motion velocity vector collation processing devicecollates the output sequence from the motion velocity vector estimationprocessing device with the output sequence from the image analysisprocessing device, the motion velocity vector collation processingdevice determines a weighting factor in accordance with a size of anarea of a unit pixel of the image of the monitoring camera projected ona floor surface at the position of the feet of the moving body estimatedfrom the image of the monitoring camera, to carry out the collationprocess. This is similarly carried out in a case in which the motionvector collation processing device collates the output sequence from themotion vector estimation processing device with the output sequence fromthe image analysis processing device. Moreover, in this case, in thecollation process, the weighting factor is made smaller as the area ofthe unit pixel is projected larger, and the weighting factor is madelarger as the area of the unit pixel is projected smaller.

Moreover, as another feature, the moving body positioning device of thepresent invention further includes a position correction signal outputdevice, when the motion velocity vector collation processing deviceoutputs the TRUE signal, the position correction signal output deviceoutputting a signal that corrects a positional coordinate of the movingbody to a position of the moving body wearing the self-contained sensorgroup, the positional coordinate being outputted from the image analysisprocessing device.

As yet another feature, the moving body positioning device of thepresent invention further includes a walking parameter estimationprocessing device, the motion velocity vector estimation processingdevice correcting and outputting the output sequence of the motionvelocity vector in accordance with an individual difference parameterset in advance, and when the motion velocity vector collation processingdevice outputs the TRUE signal, the walking parameter estimationprocessing device resetting the individual difference parameter in themotion velocity vector estimation processing device with use of pairinformation of time series data of a collated motion velocity vector andan output sequence of sensor data of the self-contained sensor.

Moreover, the moving body positioning device according to the presentinvention further includes an identification information storage/displaydevice that stores and displays identification information identifyingthe moving body, when the motion velocity vector collation processingdevice outputs the TRUE signal, the identification informationstorage/display device storing and displaying, as information indicativeof the moving body in the image of the monitoring camera, theidentification information of a moving body that wears theself-contained sensor, and when the motion velocity vector collationprocessing device outputs the FALSE signal, the identificationinformation storage/display device storing and displaying, asinformation indicative of the moving body in the image of the monitoringcamera, the identification information indicating that no self-containedsensor is worn.

Advantageous Effects of Invention

With use of the moving body positioning device of the present inventionincluding the foregoing features, when a person (moving body) wearingfor example PDR sensors (an internal observation device and aself-contained sensor group of positioning devices) enters into an imageof for example a monitoring camera (an external observation device), anestimation result of a position of the wearing person (moving body) canbe corrected based on an analysis result of that image. Hence, nomeasurement error of the position/direction caused by the PDR isaccumulated together with movement of the moving body, as like in theconventional art.

Moreover, by using the moving body positioning device of the presentinvention, in a case in which a person wearing for example the PDRsensors (an internal observation device and a self-contained sensorgroup of the positioning device) is in the image of for example themonitoring camera (an external observation device), it is possible toestimate a motion velocity of the wearing person (moving body) based onthat image. By associating this with the PDR sensor data based on thisresult, it is possible to estimate an individual difference parameter ofthe wearing person. With use of this estimated individual differenceparameter, it is possible to carry out a calibration operation inadvance.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1

FIG. 1 is a view schematically illustrating a moving body positioningdevice according to the present invention.

FIG. 2

FIG. 2 is a block diagram illustrating basic processing elements of themoving body positioning device according to the present invention.

FIG. 3

FIG. 3 is a view illustrating another embodiment of the moving bodypositioning device according to the present invention.

FIG. 4

FIG. 4 is a block diagram illustrating a configuration of anotherembodiment of the moving body positioning device according to thepresent invention.

FIG. 5

FIG. 5 is a block diagram illustrating a configuration of anotherembodiment of the moving body positioning device according to thepresent invention.

FIG. 6

FIG. 6 is a block diagram illustrating a configuration of anotherembodiment of the moving body positioning device according to thepresent invention.

FIG. 7

FIG. 7 is a flowchart describing a series of processes for setting aweighting factor in a collation process carried out by a motion velocityvector collation processing device or a motion vector collationprocessing device.

FIG. 8

FIG. 8 is a block diagram illustrating a configuration of anotherembodiment of the moving body positioning device of the presentinvention.

FIG. 9

FIG. 9 is a block diagram illustrating a configuration of anotherembodiment of the moving body positioning device of the presentinvention.

FIG. 10

FIG. 10 is a block diagram illustrating a configuration of anotherembodiment of the moving body positioning device of the presentinvention.

DESCRIPTION OF EMBODIMENTS

Described below is an embodiment of a moving body positioning device ofthe present invention, with reference to drawings. FIG. 1 is a viewschematically illustrating a moving body positioning device according tothe present invention, and FIG. 2 is a block diagram illustrating aconfiguration of basic processing elements of the moving bodypositioning device according to the present invention.

An external observation device 11 in FIG. 1 is a device that can observemovements of a plurality of moving bodies in a predetermined space(external world) such as a road, an open space, a surface of the oceanor the like. Examples of the external observation device 11 include amonitoring camera, a Z-value sensor, a laser range finder and the like.An internal observation device 12 is a device worn on a moving body, andmeasures motion of the moving body. Examples of the internal observationdevice 12 are combinations of (i) a sensor that measures movement suchas an acceleration sensor, a magnetic sensor, an angular velocitysensor, or a pressure sensor and (ii) a clock. As the sensor thatmeasures movement, a temperature sensor may be combined with theacceleration sensor, the magnetic sensor, the angular velocity sensor,or the pressure sensor. Examples of the moving body encompass apedestrian, a bicycle, an automobile, an airplane, and a ship.

The external observation device 11 observes each of moving bodieswearing the internal observation device 12 (in the example of FIG. 1, amoving body 2 being a pedestrian) and a moving body 1 (similarly in thisexample, a pedestrian) not wearing the internal observation device 12 ornot sending out information of the internal observation device 12although the moving body is wearing the internal observation device 12.At this time, movement of each of the plurality of moving bodies that isto be observed is estimated for every discrete time based on a motionvelocity vector, motion vector, motion identification and the like.

For example, when the external observation device 11 is a monitoringcamera that image captures a moving body, the movement of the movingbody can be estimated by tracing the moving body in the image of themonitoring camera as appropriate by its shape, color and the like, andcombining this with a motion velocity vector estimation processingdevice that estimates a motion velocity vector of the moving body.

Moreover, the movement of the moving body can be estimated by use of asensor that measures a depth (Z value) of the external world as theexternal observation device 11, and combining this with a Z-valueanalysis device that detects a moving body which is the subject to beimage captured and estimates its motion velocity vector depending on itsdepth. The sensor for measuring the Z value may be a depth measurementdevice using a stereo camera in a case of a passive sensor, and may be adevice which measures a distance (Z-value) by emitting infrared lightand measuring a time required for the infrared light to reflect back, ina case of an active sensor.

Alternatively, the movement of the moving body can be estimated by usinga Laser Range Finder (LRF) as the external observation device 11, whichLRF emits a laser beam to the external world and measures a distancefrom the moving body based on its reflected light, and combine therewithan analysis device that detects the moving body based on the measurementresult of that distance and estimates its motion velocity vector.

Meanwhile, the internal observation device 12 estimates the movement ofthe moving body that wears the internal observation device 12, for everydiscrete time based on the motion velocity vector, the motion vector,the motion identification and the like. Time series data of the motionvelocity vector observed in each of the external world and the internalworld is collected to one location by communications means (e.g., withuse of wireless data communications network). At that location, acollation process (A) is carried out to the time series data of themotion velocity vector of at least one of the moving bodies that theexternal observation device 11 captures, with the motion velocity vectoroutputted by the internal observation device 12, and when the collationis successful and is deemed as a time series of a motion velocity vectorof an identical moving body, a TRUE signal is outputted, and for anyother cases, a FALSE signal is outputted.

FIG. 2 is a block diagram illustrating a configuration of basicprocessing elements in the moving body positioning device according tothe present invention. The external observation device 21 is a devicesimilar to the external observation device 11 of FIG. 1, and detects atleast one moving body and outputs its result together with time data.Output data 30 differs depending on a used external observation device;for example, when a camera is used, the output data 30 is image datawith time, when the Z-value sensor is used, the output data 30 is acollection of Z values with time, and when the laser range finder isused, the output data 30 is a collection of distance data with time. Anexternal observation data processing device 24 is a device that detectsa moving body based on the output data 30 and outputs time series dataof its motion velocity vector, motion vector, or motion identificationresult of the moving body; more specifically, is a computer including amemory, a processor, and an input/output interface. This can be achievedby, for example, a process of detecting a subject moving body (e.g. ahuman figure) based on features such as shape and color, in imageanalysis.

The internal observation device 22 is, as illustrated in

FIG. 1, a device that is worn on a moving body to measure its motion,similarly to the internal observation device 21 of FIG. 1. The internalobservation device 22 is achievable by, for example, use of aself-contained sensor (a combination of acceleration, magnetic, angularvelocity, pressure, and temperature sensor) with a clock (real-timeclock, etc.). The internal observation data processing device 23 is adevice which, in a case in which the motion velocity vector is to beestimated, obtains an acceleration vector fixed to a (three-dimensional)world coordinate system based on a time-attached output from theself-contained sensor, and estimates the motion velocity vector by itsacceleration integral, and more specifically is a computer having amemory, a processor, and an input/output interface. Moreover, in a casein which the moving body is limited to just pedestrians, it isachievable by estimating the motion velocity vector by the method of PDR(Pedestrian Dead Reckoning). The internal observation data processingdevice 23 outputs time series data (33) of the motion velocity vector ormotion vector of the moving body wearing the internal observation device22, or a motion identification result of the moving body.

Finally, the collation processing device 25 collates (a) the time seriesdata (31) of the motion velocity vector, motion vector, or motionidentification result of the moving body related to one moving bodyobtained based on the external observation device 21 with (b) the timeseries data (33) of the motion velocity vector, motion vector or motionidentification result of the moving body obtained based on the internalobservation device 22, and in a case in which the two can be collated,the collation processing device 25 outputs a TRUE signal, and in othercases, outputs a FALSE signal (34). Moreover, the collation processingdevice 25 is also a computer including a memory, a processor, and aninput/output interface, and carries out the collation process of datareceived from the external observation data processing device and theinternal observation data processing device with use of the processor.

For example, in a case of collating the time series data of the motionvelocity vector, a distance scale between the time series data of themotion velocity vectors is set beforehand based on the time series data(31) of the motion velocity vector and the time series data (33) of themotion velocity vector obtained in accordance with the internalobservation device 22; when this distance scale is not more than a setthreshold, the two motion velocity vectors are determined as beingcollated and thus a TRUE signal is outputted, whereas in any other case,a FALSE signal is outputted (34). Note that, as the distance scaledescribed before, a distance scale that allows for a certain jitter(time difference) to be present in the time data held by the two timeseries data is selected and used.

FIG. 3 illustrates a state in which a monitoring camera 101 is providedin a known disposition relationship with the floor surface, and theimage captured by the monitoring camera 101 includes a path, an openspace or the like where people often pass. The self-contained sensor 102is a collection of sensors that can operate without externalinfrastructure; as the self-contained sensor 102, an accelerationsensor, a gyro sensor, a magnetic sensor, a pressure sensor or the likemay be used for example.

Here, with the monitoring camera 101, when an environment model isgenerated upon appropriately setting a scale, a translation and rotationmovement parameter with respect to an external environment in which themonitoring camera 101 is provided and a camera parameter constituted ofa focus distance and scale coefficient are found by carrying out thefollowing procedures. Such an environment model can be generatedinteractively by executing a modeler application program (modelerapplication) with use of a method described in Non Patent Literature 3or the like. As described in Non Patent Literature 3, procedures arecarried out for finding a vanishing point in the environment based on aninstruction by the user, with use of the modeler application describedbefore (more specifically, the user selects a pair of two parallelstraight lines in the environment).

FIG. 3 illustrates a state in which a pedestrian, who is a moving bodywearing the self-contained sensor 102, is detected as being included andmoving in the image of the monitoring camera 101, and a process (A) ofcollating the human figure in the image of the monitoring camera 101with the pedestrian wearing the self-contained sensor 102 is carriedout.

In FIG. 4, 201 is a monitoring camera, 202 is a self-contained sensorgroup, 203 is a motion velocity vector estimation processing device, 204is an image analysis processing device, and 205 is a motion velocityvector collation processing device. With the configuration includingthese processing components, the image of the monitoring camera iscollated with the output of the self-contained sensor, to output a TRUEor a FALSE signal.

The image (frame image of each time) captured by the monitoring camera201 is outputted to the image analysis processing device 204. Here,candidates of figures of the human figure of the moving body isselected, positions of their feet are estimated, and motion velocityvector of each time is estimated by time integrals of the positions, tooutput an output sequence of the motion velocity vector. Note that theestimation of the motion velocity vector can be carried out by obtainingdata with, instead of the monitoring camera, a depth measurement devicewith use of a stereo camera, or a laser range finder. Hereinafter, themoving body is described as a human figure.

In the embodiment, 210 is information of each of frame images of theimage of the monitoring camera, and 211 is information of a result ofanalyzing the frame images. In a case in which a human figure is presentin the image, data indicative of a positional coordinate of a floorsurface on which the feet of the human figure is present is outputted asthe information 211 of the analysis result, and in a case in which nohuman figure is included, data indicative of a signal of that fact isoutputted as the information 211 of the analysis result.

On the other hand, the output data 212 of the self-contained sensorgroup 202 is outputted to the motion velocity vector estimationprocessing device 203; based on the output data (acceleration vector,angular velocity vector, magnetic vector, pressure data) of theself-contained sensor group 202, the motion velocity vector of thewearing person (human figure) for each of the times is estimated.

The output data 212 is, as described above, an output of sensor datafrom the sensor included in the self-contained sensor group 202. Theself-contained sensor group 202 includes a timer, and an accelerationvector, angular velocity vector, magnetic vector, and pressure data,each having time stamp information obtained by the timer, are outputtedfrom the self-contained sensor group 202. Calculation of the motionvelocity vector based on the sensor data of the output data 212 of theself-contained sensor group 202 is, for example, carried out byprocessing data in the method described in Non Patent Literature 1. Data213 outputted from the motion velocity vector estimation processingdevice 203 is data of an output sequence of a motion velocity vector ofa moving body (human figure) that wears the self-contained sensor group202.

The motion velocity vector collation device 205 accumulates the outputsequences of the motion velocity vector outputted from the imageanalysis processing device 204 and the output sequences of the motionvelocity vector outputted from the motion velocity vector estimationprocessing device 203 for a certain time and compares the two, todetermine whether or not these output sequences match each other. Thedata processing of this determination is, as disclosed in Non PatentLiterature 4, a data processing in which a total of magnitudes of adifference vector between the two motion velocity vectors are calculatedand thereafter this total is normalized with a length of time or asample number; the normalized total is compared with a predeterminedthreshold, and is determined as being collated in the case in which thenormalized total is not more than the threshold.

The time series data of the motion velocity vector estimated by themotion velocity vector estimation processing device 203 based on thesensor data of the self-contained sensor group 202 is collated with themotion velocity vector estimated from the positional coordinates of thefeet of the human figure in the image obtained as an analysis result ofthe frame images from the monitoring camera 201 by the image analysisprocessing device 204, and as its result, in a case in which the twocollate with each other, a signal indicative of TRUE is outputted as theoutput data 214 outputted from the motion velocity vector collationdevice 205, and in a case in which the two do not collate with eachother, a signal indicative of FALSE is outputted as the output data 214outputted from the motion velocity vector collation device 205.

The moving body positioning device of the embodiment described abovecarries out the collation process based on the output sequence of themotion velocity vector, however it is also possible to carry out thecollation process based on a motion vector (i.e. relative positionalvector) instead of collating with the motion velocity vector. Anembodiment with such a configuration is described below.

FIG. 5 is a block diagram describing a configuration of anotherembodiment of the moving body positioning device of the presentinvention. In FIG. 5, 301 is a monitoring camera, 302 is aself-contained sensor group, 303 is a motion vector estimationprocessing device, 304 is an image analysis processing device, and 305is a motion vector collation processing device. The monitoring camera301 is as with the monitoring camera 201 in FIG. 4. The monitoringcamera 301 outputs a frame image signal 310. The image analysisprocessing device 304, upon receiving the frame image signal 310 anddetermining that a human figure (moving body) is present inside theframe image, outputs an output sequence of the motion vector 311 of thathuman figure.

On the other hand, the self-contained sensor group 302 is as with theself-contained sensor group 202 in FIG. 4. The output data 312 outputtedfrom the self-contained sensor group 302 is an output sequence of sensordata such as an acceleration vector, an angular velocity vector, amagnetic vector, and pressure data, each to which a time stamp is added.The motion vector estimation processing device 303, when receiving suchsensor data of the output data 312 from the self-contained sensor group302 as its input, estimates and outputs a relative motion vector of aperson (moving body) wearing the self-contained sensor based on dataprocessing such as carrying out integral processing of a velocityvector. For such estimation processing of a motion vector, it ispossible to utilize, for example, data processing of calculating themotion vector by estimating a motion velocity and carrying out firstorder integral related to that time, as described in Non PatentLiterature 1, and so thus the data processing of this method is used.The motion vector estimation processing device 303 outputs output data313 of the output sequence of the calculated relative motion velocityvector.

The motion vector collation processing device 305 collates the timeseries data of the output data 311 of the motion vector obtained by theimage analysis processing with the time series data of the output data313 of the relative motion vector obtained by the motion vectorestimation processing based on the sensor data. The time series data ofthe two motion vectors are collated by performing a collation process inwhich a total of magnitudes of difference vectors between the two motionvectors is calculated and its total is normalized with a length of timeor a sample number, thereafter the normalized total is compared with apredetermined threshold and is determined as being collated in a case inwhich the normalized total is not more than that threshold. As a resultof such a process, the motion vector collation processing device 305outputs, as the output data 314, a TRUE signal in a case in which it isdetermined as collated, and a FALSE signal in any other cases.

In the image analysis processing of the image analysis processing device304 in this case, a detection accuracy of a position based on an imageof the monitoring camera 301 when a human figure is included in theimage as a small size decreases as compared to a case in which the humanfigure is included in the image as a large size. This means that when aunit pixel of the monitoring camera is back projected on the floorsurface, the reliability changes in inverse proportion to its area.Hence, the motion vector collation processing device 305, when theoutput sequence from the motion vector estimation processing device 303collates with the output sequence from the image analysis processingdevice 304, carries out the collation processing upon determining aweighting factor in accordance with an area of the unit pixel of theimage of the monitoring camera projected on the floor surface at theposition of the feet of the moving body estimated from the image of themonitoring camera.

In this case, in the collation processing, the weighting factor is madesmaller as the area of the unit pixel is projected larger, and theweighting factor is made greater as the area of the unit pixel isprojected smaller. Such a process can be similarly carried out in theembodiment of the moving body positioning device described withreference to FIG. 4. Namely, in this case, the motion velocity vectorcollation processing device 205, when the output sequence from themotion velocity vector estimation processing device 203 is collated withthe output sequence from the image analysis processing device 204,carries out the collation upon determining the weighting factor inresponse to an area of the unit pixel of the image of the monitoringcamera projected on the floor surface of the position of the feet of themoving body estimated from the image of the monitoring camera.

FIG. 6 is a block diagram describing a configuration of anotherembodiment of the moving body positioning device of the presentinvention. The embodiment described in FIG. 6 is an embodiment in which,based on observation data by the internal observation device andexternal observation device, a self-contained base movement kindprocessing device, which is the internal observation data processingdevice, and an image analysis processing device, which is the externalobservation data processing device, identify a movement kind of themoving body, and collates the time series data of the identificationresult in the collation processing device.

In FIG. 6, 3201 is a monitoring camera, 3202 is a self-contained sensorgroup, 3203 is a self-contained base movement kind processing device,3204 is an image analysis processing device (motion type processingdevice), and 3205 is a movement kind collation processing device. Themonitoring camera 3201 corresponds to the external observation device 21in FIG. 2, and is as with the monitoring cameras 201 and 301. Themonitoring camera 3201 outputs a time-attached image signal 3210. Theimage analysis processing device (movement kind processing device) 3204receives the time-attached image signal 3210 as input, analyzes themovement of the moving body (human figure or the like) included in theimage, and recognizes and identifies movements thereof. Several methodsof known image processing and computer vision such as HMM (Hidden MarkovModel) are available as methods of recognizing the movement kinds of thehuman figure, and recognition may be achieved by use of this method.Upon identifying the movement kind of the moving body, itsidentification result 3211 of the movement kind is outputted.

On the other hand, the self-contained sensor group 3202 corresponds tothe internal observation device 22 in FIG. 2, and is as with theself-contained sensor groups 202 and 302. The output data 3212 outputtedfrom the self-contained sensor group 3202 is an output sequence ofsensor data such as an acceleration vector, an angular velocity vector,a magnetic vector, pressure data, temperature data, each to which a timestamp is attached.

The self-contained base movement kind processing device 3203 identifiesand recognizes a movement kind of the person wearing the self-containedsensor group 3202, based on the output data 3212. Such a movement kindprocessing device, for example, calculates (i) features vector of a timeregion of time series data such as the acceleration vector and theangular velocity vector that are part of the output from theself-contained sensor group, and (ii) features vector in a frequencyregion of the time series data converted by Fourier transformation. Themovement kinds corresponding to the combination of the two featuresvectors, of the time region and of the frequency region, can beidentified and recognized by a method of machine learning with use of acalculator. Examples of frameworks of the machine learning includeAdaBoost (Adaptive Boosting), DP (Dynamic Programming) matching, and SVM(Support Vector Machine); by collating the received time series sensordata with learnt models or role model data with use of results of themachine learning of the data collected beforehand or actively collectedby such a framework, it is possible to add to the input sensor data aspecific identification result. By use of this method, it is possible tooutput (3213) time series data of an identification result of themovement kind.

The movement kind collation processing device 3205 corresponds to thecollation processing device 25 in FIG. 2, and collates time series dataof the two movement kind recognition results based on the output 3211 ofthe motion type processing device 3204 and the output 3213 of theself-contained base movement kind processing device 3203. At this time,a distance scale between the time scale movement kind recognition resultdata is defined, and when this distance is not more than a setthreshold, a TRUE signal is outputted as the two time series data beingcollated to each other, and in any other case, a FALSE signal isoutputted (3214).

FIG. 7 is a flow chart describing processes of setting a weightingfactor in the collation process carried out by the motion velocityvector collation processing device or the motion vector collationprocessing device. These processes are described below, with referenceto FIG. 7. In this process, a position on the floor surface on which thehuman figure is present is found in the image of the monitoring camerain step S401, and subsequently in step S402, an area of which the unitpixel of the monitoring camera is projected on the floor surface onwhich the human figure is present is calculated. Thereafter, in thesubsequent step S403, the weighting factor at that position isdetermined in accordance with the area calculated, and in step S404, theweighting factors for each of points on its trail is stored. Theweighting factors are determined as such, to carry out the collationprocess.

Moreover, the moving body positioning device of the present inventionmay be modified so that an individual difference parameter is set,thereby allowing for improving the accuracy of the data processing inthe estimation process. In this case, when the image of the monitoringcamera is collated with the output of the self-contained sensor, theindividual difference parameter is set by estimating an individualdifference parameter that would characterize a walking movement of thehuman figure wearing the self-contained sensor, with use of the motionvelocity vector of the human figure in the image and the output sequenceof the self-contained sensor.

When the motion velocity vector or the motion vector of the human figureinside the image of the monitoring camera is associated with the motionvelocity vector or the motion vector estimated in accordance with sensordata based on the self-contained sensor group, it is possible toassociate the output sequence of the motion velocity vector of the humanfigure in the image of the monitoring camera with the output sequence ofthe sensor data of the self-contained sensor group. Accordingly, as alsodescribed in Non Patent Literature 1, for example, amplitude data of anacceleration component obtained by decomposing the output of theacceleration sensor in a vertical direction is associated with amagnitude of the motion velocity vector of the human figure in theimage, to estimate an individual difference parameter (in theembodiment, a slope and intercept that determine a straight line) ofthat human figure (i.e. person wearing the self-contained sensor group).

Such an embodiment is described with reference to FIG. 8. FIG. 8 is ablock diagram illustrating a configuration of another embodiment of themoving body positioning device according to the present invention. InFIG. 8, 201 is a monitoring camera, 202 is a self-contained sensorgroup, 203 is a motion velocity vector estimation processing device, 204is an image analysis processing device, and 205 is a motion velocityvector collation processing device. These components are as with thosedescribed with reference to FIG. 2. In the embodiment, the configurationfurther includes a walking parameter estimation processing device 505.

The walking parameter estimation processing device 505 receives thesensor data 503 outputted from the self-contained sensor group 202, themotion velocity vector 501 of the position of the feet of the humanfigure inside the image, outputted from the image analysis processingdevice 204, and further the output signal 502 of a collation result(TRUE/FALSE) outputted as a result of the collation process by themotion velocity vector collation processing device 205, and estimatesand outputs a walking parameter. The estimated output of the walkingparameter is inputted into the motion velocity vector estimationprocessing device 203, and is used when the motion velocity vectorestimation processing device 203 carries out the estimation process ofthe motion velocity vector.

Namely, the process of estimating the walking parameter carried out bythe walking parameter estimation processing device 505 is a process inwhich, when the motion velocity vector of the human figure in the imageis collated with the motion velocity vector based on the sensor data ofthe self-contained sensor (i.e. when the signal 502 is a TRUE signal),the walking parameter estimation processing device estimates, based onthe sensor data 503 of the self-contained sensor data and the motionvelocity vector 501 associated at that time, a walking parametercharacterizing the walking movement of the person wearing theself-contained sensor, and outputs the data of that walking parameter.The outputted walking parameter characterizing the individual differenceis, as described above, used when the motion velocity vector estimationprocessing device 203 carries out the estimation process of the motionvelocity vector.

The walking parameter estimation processing device 505 provided in themoving body positioning device illustrated in FIG. 8 resets the walkingparameter of the individual difference parameter in the motion velocityvector estimation processing device 203 when the motion velocity vectorcollation processing device 205 outputs a TRUE signal as the output data214 of the collation process, with use of pair information of timeseries data of the collated motion velocity vector and the outputsequence of sensor data of the self-contained sensor. Hence, the motionvelocity vector estimation processing device 203 corrects and outputsthe output sequence of the motion velocity vector, in accordance withthe walking parameter of the set individual difference parameter.

Moreover, the moving body positioning device according to the presentinvention may be modified so that when the image of the monitoringcamera is collated with the output of the self-contained sensor,identification information of the person wearing the self-containedsensor is associated with the human figure in the image, and thisidentification information is stored and displayed. An embodiment ofsuch a configuration is described with reference to FIG. 9.

FIG. 9 is a block diagram illustrating a configuration of anotherembodiment of the moving body positioning device of the presentinvention. In FIG. 9, 201 is a monitoring camera, 202 is aself-contained sensor group, 203 is a motion velocity vector estimationprocessing device, and 205 is a motion velocity vector collationprocessing device. Furthermore, 601 is a wearing person identificationinformation storage device, 604 is an identification informationstorage/display device, and 605 is an image analysis processing device.

The wearing person identification information storage device 601 is adevice that stores and outputs identification information (informationsuch as an individual ID and name) of a person that wears theself-contained sensor group. An output 603 of the identificationinformation is identification information of the person wearing theself-contained sensor group 202. The image analysis processing device605 here analyzes an image from the monitoring camera 201, estimates amotion velocity vector of a position of the feet of the human figure,and outputs its output data 211. Simultaneously, the image analysisprocessing device 605 outputs output data 602 of region information inwhich the human figure is included.

The output data 602 of the region information is information indicativeof a region of the human figure in the image. Moreover, 606 is outputdata 606 of a frame image of an image taken out from the output data 210outputted from the monitoring camera 201, and is inputted into theidentification information storage/display device 604.

The identification information storage/display device 604 receives, asinput, the frame image 606, the region information 602 of the humanfigure in the image, the identification information 603 of a personmounting the self-contained sensor group 202, and a TRUE/FALSE signal214 which is a result of a collation determination process of the twomotion velocity vectors, which result is the output of the motionvelocity vector collation processing device 205. When the signal of theresult of the collation determination process is the TRUE signal, theregion information 602 of the frame image 606 of the monitoring camerais stored associated with the identification information 603 of thewearing person, and the identification information 603 is displayed onthe image region indicated by the region information in the frame image.

As a result, in the moving body positioning device according to thepresent invention, when the motion velocity vector or motion vector ofthe human figure in the image of the monitoring camera is associatedwith the motion velocity vector or motion vector estimated based on thesensor data according to the self-contained sensor group, the humanfigure in the image of the monitoring camera may be associated with theidentification information of the person wearing the self-containedsensor group, and display these information.

The moving body positioning device of the embodiment includes anidentification information storage/display device 604 that stores anddisplays identification information identifying a moving body; when themotion velocity vector collation processing device 205 outputs a TRUEsignal, the identification information storage/display device 604 storesand displays identification information of a moving body that wears theself-contained sensor 202 as information indicative of the moving bodyin the image of the monitoring camera 201, and when the motion velocityvector collation processing device 205 outputs a FALSE signal, theidentification information storage/display device 604 stores anddisplays identification information indicating that no self-containedsensor is worn, as the information indicative of the moving body in theimage of the monitoring camera 201.

Moreover, in the moving body positioning device according to the presentinvention, when the motion velocity vector or motion vector of the humanfigure in the image of the monitoring camera is associated with a motionvelocity vector or motion vector estimated based on the sensor dataaccording to the self-contained sensor group, it is possible to estimatewith a certain probability that a person wearing the self-containedsensor group is present at a position on which a human figure ispresent, based on an image analysis of the monitoring camera.

In this case, when the image of the monitoring camera is collated withthe output of the self-contained sensor, the position of the personwearing the self-contained sensor group is corrected to the position ofthe image analysis result. The position correction signal used at thistime is outputted from the position correction signal output device 702.

FIG. 10 is a block diagram illustrating a configuration of anotherembodiment of the moving body positioning device of the presentinvention. In FIG. 10, 201 is a monitoring camera, 202 is aself-contained sensor group, 203 is a motion velocity vector estimationprocessing device, 204 is an image analysis processing device, and 205is a motion velocity vector collation processing device. Thesecomponents are configured similarly to the configuration of theembodiment of the moving body positioning device described withreference to FIG. 2. The 702 is a position correction signal outputdevice.

The position correction signal output device 702 receives, as its input,positional information 701 of the feet of the human figure in the image,which is one output of the image analysis processing device 204, and asignal (TRUE/FALSE signal) 214 of a collation result, which is an outputof the motion velocity collation processing device 205, and outputs theposition correction signal 703. The position correction signal 703 isused as a correction signal for an image analysis signal 701 outputtedfrom the image analysis processing device 204. The position correctionsignal output device 702, when the motion velocity vector collationprocessing device 205 outputs a TRUE signal, outputs a positionalcoordinate of the moving body that is outputted from the image analysisprocessing device 204 as a signal for correcting as a position of themoving body on which the self-contained sensor group is worn.

INDUSTRIAL APPLICABILITY

A moving body positioning device of the present invention allows forcorrecting a position estimation result obtained by an internalobservation device provided to a moving body with use of an externalobservation device such as a monitoring camera, thereby making itpossible to measure, analyze, and estimate movement of a moving bodymore accurately. Accordingly, the moving body positioning device of thepresent invention is useful in an industrial manner.

REFERENCE SIGNS LIST

11 external observation device

12 internal observation device

21 external observation device

22 internal observation device

23 internal observation data processing device

24 external observation data processing device

25 collation processing device

101 monitoring camera

102 self-contained sensor group

201 monitoring camera

202 self-contained sensor group

203 motion velocity vector estimation processing device

204 image analysis processing device

205 motion velocity vector collation processing device

301 monitoring camera

302 self-contained sensor group

303 motion vector estimation processing device

304 image analysis processing device

305 motion vector collation processing device

505 walking parameter estimation processing device

601 wearing person identification information storage device

604 identification information storage/display device

605 image analysis processing device

702 position correction signal output device

3201 monitoring camera

3202 self-contained sensor group

3203 self-contained base movement kind processing device

3204 image analysis processing device

3205 movement kind collation processing device

1. A moving body positioning device, comprising: an internal observationdevice provided to a moving body, to measure and output a motion of themoving body; an internal observation data processing device thatestimates a motion of the moving body based on the output from theinternal observation device; an external observation device thatobserves motion of a plurality of moving bodies; an external observationdata processing device that estimates a motion of the moving body froman observation result by the external observation device; and acollation processing device that collates an output from the internalobservation data processing device with that of the external observationdata processing device, and outputs its collation result as a TRUEsignal or a FALSE signal.
 2. The moving body positioning device recitedin claim 1, wherein the internal observation device includes aself-contained sensor group that outputs a motion velocity vector of themoving body with use of an acceleration sensor provided in the movingbody, the internal observation data processing device is a motionvelocity vector estimation processing device that measures a motionvelocity vector based on an output of the self-contained sensor group,and outputs an output sequence of the motion velocity vector togetherwith a measured time, the external observation device includes amonitoring camera for externally image capturing the moving bodies, theexternal observation data processing device analyzes an image of themonitoring camera to measure a position of feet of the moving body inthe image to measure a motion velocity vector of that position of thefeet, and outputs an output sequence of the motion velocity vectortogether with a measured time, and the collation processing device is amotion velocity vector collation processing device that collates theoutput sequence of the motion velocity vector estimation processingdevice with the output sequence of the image analysis processing device,and outputs its collation result as a TRUE signal or a FALSE signal. 3.The moving body positioning device recited in claim 1, wherein theinternal observation device includes a self-contained sensor group thatoutputs a motion vector of the moving body, the motion vector beingobtained by integrating a motion velocity vector with use of anacceleration sensor provided in the moving body, the internalobservation data processing device is a motion vector estimationprocessing device that measures a motion vector based on an output ofthe self-contained sensor group and outputs an output sequence of themotion vector together with a measured time, the external observationdevice includes a monitoring camera for externally image capturing themoving bodies, the external observation data processing device is animage analysis processing device that analyzes an image of themonitoring camera to measure a position of feet of the moving body inthe image to measure a motion vector of that position of the feet, andoutputs an output sequence of the motion vector together with a measuredtime, and the collation processing device is a motion velocity vectorcollation processing device that collates the output sequence of themotion vector estimation processing device with the output sequence ofthe image analysis processing device, and outputs its collation resultas a TRUE signal or a FALSE signal.
 4. The moving body positioningdevice recited in claim 1, wherein the internal observation deviceincludes a self-contained sensor group that outputs a motion vector ofthe moving body, the motion vector being obtained by integrating amotion velocity vector with use of an acceleration sensor provided inthe moving body, the internal observation data processing device is aself-contained sensor base motion identification processing device thatidentifies a movement kind of the moving body in accordance with anoutput of the self-contained sensor group, and outputs an identificationresult together with a measured time, the external observation deviceincludes a monitoring camera for externally image capturing the movingbodies, the external observation data processing device is an imageanalysis processing device that analyzes an image of the monitoringcamera to identify the movement kind of the moving body in the image andoutputs its identification result together with a measured time, and thecollation processing device is a movement kind collation processingdevice that collates the output of the self-contained sensor base motionidentification processing device with the output of the image analysisprocessing device, and outputs its collation result as a TRUE signal ora FALSE signal.
 5. The moving body positioning device recited in claim2, wherein when the collation processing device collates the outputsequence from the internal observation data processing device with theoutput sequence from the external observation data processing device,the collation processing device determines a weighting factor inaccordance with a size of an area of a unit pixel of the image of themonitoring camera projected on a floor surface at the position of thefeet of the moving body estimated from the image of the monitoringcamera, to carry out the collation process.
 6. The moving bodypositioning device recited in claim 5, wherein in the collation process,the weighting factor is made smaller as the area of the unit pixel isprojected larger, and the weighting factor is made larger as the area ofthe unit pixel is projected smaller.
 7. The moving body positioningdevice recited in claim 2, further comprising: a position correctionsignal output device, when the collation processing device outputs theTRUE signal, the position correction signal output device outputting asignal that corrects a positional coordinate of the moving body to aposition of the moving body wearing the self-contained sensor group, thepositional coordinate being outputted from the external observation dataprocessing device.
 8. The moving body positioning device recited inclaim 2, further comprising: a walking parameter estimation processingdevice, the internal observation data processing device correcting andoutputting the output sequence of the motion velocity vector inaccordance with an individual difference parameter set in advance, andwhen the collation processing device outputs the TRUE signal, thewalking parameter estimation processing device resetting the individualdifference parameter in the internal observation data processing devicewith use of pair information of time series data of a collated motionvelocity vector and an output sequence of sensor data of theself-contained sensor.
 9. The moving body positioning device recited inclaim 2, further comprising: an identification informationstorage/display device that stores and displays identificationinformation identifying the moving body, when the collation processingdevice outputs the TRUE signal, the identification informationstorage/display device storing and displaying, as information indicativeof the moving body in the image of the monitoring camera, theidentification information of a moving body that wears theself-contained sensor, and when the collation processing device outputsthe FALSE signal, the identification information storage/display devicestoring and displaying, as information indicative of the moving body inthe image of the monitoring camera, the identification informationindicating that no self-contained sensor is worn.